The standards for a successful evaluation, including utility, accuracy, feasibility, propriety, and accountability (Alkin, 2010) reflect the values indicated by our stakeholders in a number of ways. Of primary importance to our stakeholders was that the results of this evaluation have utility; that is, that the results be relevant and help them to implement program improvements. Throughout this participatory evaluation, primary stakeholders and evaluators monitored the accuracy of data. Evaluation conclusions and decisions were jointly arrived at through a dialogic process, thus ensuring sensitivity to the culture and context in which they would be applied. The evaluation team jointly ensured feasibility of the study, as they collaboratively developed a plan that would work within the unique setting of L. B. Primary School. Existing practices and knowledge (such as familiarity with the DRA, and having teachers administer assessments) were built into the design, demonstrating sensitivity and responsiveness to the way in which the program was currently operating. Issues of propriety were addressed through a participatory evaluation design that was responsive to our stakeholders at all stages of the process. The evaluation agreement took into account the needs and expectations of our stakeholders and was arrived at through a process of negotiation. Finally, this evaluation was fully documented, including negotiated purposes, design, procedures, data, and outcomes, thus ensuring full accountability (Alkin, 2010, citing Yarbrough, Shulha, Hopson, & Caruthers).
Analysis and Synthesis
The analysis and synthesis of the evaluation findings reflect rigorous examination of the data through the emergence of multiple sources of corroborating evidence. By using various qualitative and quantitative measures, the evaluators were able to collect data and proceed with confidence that it reflected the true state of the L.B. Primary School Guided Reading program. Through the collection, analysis, and interpretation of triangulated data, the evaluators were able to depict and support with evidence the dominant themes that emerged, such as the infidelity of program implementation, time constraints, lack of assessment driven instruction, poor accessibility to resources and training, and a lack of student engagement. Our evaluation conclusions were based on multiple sources of confirmation, providing a detailed and comprehensive picture of the program, and contributing to the trustworthiness of the study.
Interpretation
With our analysis and synthesis of the evaluation findings in mind, the evaluation team reexamined the questions that guided this program.
We examined our over arching question: Is Guided Reading Effective?
The evaluation team also examined the sub questions, which were integral to the program evaluation
Is the GR program at L. B. Primary School being delivered as intended
and outlined by Fountas & Pinnell (1996)
Are students who participate in the program demonstrating improvements
in reading skills over the course of the school year?
Based on the evidence of the program evaluation and in collaboration with our stakeholders, we concluded that the GR program at L. B. Primary School is not being implemented with fidelity and some students are improving while others are not. The practical significance of what has been learned is that it will be necessary to take steps to improve the guided reading program at L. B. Primary School in order to make it more effective. The stakeholders will be most affected by these means of improvement. After meeting with stakeholders, it was evident that they are wanting practical solutions so that the Guided Reading Program will be effective. Stakeholders want to feel empowered and successful by delivering a guided reading program effectively and faithfully. They want students to improve academically.
Judgments
Fountas and Pinnell state that GR leads to independent reading and gives children an opportunity to grow as individual readers. It provides the readers with an opportunity to develop and apply reading strategies so that they can independently read increasingly difficult texts and to read for meaning. Guided reading develops skills needed for independent reading, and assists in teaching students to learn how to introduce texts to themselves. It is part of a balanced literacy program which measures fluency and reading comprehension by using diagnostic and developmental reading assessments, daily observations, and gathering of anecdotal records. Teachers gather observational data over time to document progress which provides information to guide daily teaching and tracks the progress of individual students. This information is a basis for reporting to parents, helps a school staff to assess the effectiveness of the instructional program, and provides children with evidence of their growth. Guided reading also encourages home and community involvement by assisting parents to participate in the school curriculum, allowing opportunities for children to show their families what they are learning, and increasing reading opportunities for students.
The evaluation indicated that this program did not achieve the intended results due to various contributing factors. Some of the factors that prevented the program from reaching the desired success included staff members lacking the time and skills to deliver the program, limited funds to purchase leveled texts and other materials to allow for the continuation of the program, and assessment data not being used regularly. Overall, teachers felt dissatisfied with the program and therefore were reluctant to continue. However, this does not indicate that the staff was solely responsible for not achieving the intended results. The results from the assessments performed indicated that some students who participated in the GR program did demonstrate improvement in reading skills over the course of the year, while others did not.
Therefore, although the program fell below the established standards, those involved in its delivery felt that it could be extremely effective in meeting the long-term goal of improving student reading skills if it was implemented with fidelity.
Recommendations The evaluation team met with stakeholders and Board staff to discuss the recommendations prior to publication. The team asked the stakeholders if the recommendations were feasible. The stakeholders were concerned about the students’ limited progress therefore they requested the evaluation even though they were hostile to the process. The stakeholders' goal is for student achievement to improve therefore they were receptive to the recommendations. During the discussion, the stakeholders made a few recommendations based on the evaluation team’s findings. The agreed upon recommendations are presented below. Based on our findings from the DRA, DIBELS, teacher interviews, and student focus groups, we propose the following recommendations:
The team saw evidence of program materials in the school but the sets were incomplete. The team realizes that the GR program cannot be implemented without the proper resources. The administration and Board office must allocate appropriate funds to replenish the missing resources.
The teachers and the student focus groups indicated to the evaluation team that the resources have to be engaging to the students. A committee should be created to purchase those resources and subsequently manage the collection, which should be kept current and compete.
The teachers indicated to the evaluation team that there is a need for GR in-service. It has been years since any of the staff obtained any in-servicing. The staff believes that in-servicing would improve delivery of GR, therefore student achievement would improve. Continual and regular in-servicing is required to sustain the GR program. The administration and board staff must support in-servicing with time and money. The staff has agreed to devote their Professional Learning Communities (Glaude, 2005) time to discuss GR.
The evaluation team and stakeholders recognize the substandard scores of the DRA and DIBELS results. When GR was first implemented, the staff received training in delivering the assessment tools but the staff did not analyze the data to make changes in program or inform instruction. The administration has agreed to provide the staff of L. B. Primary School with professional development time to analyze the DRA and DIBELS results. The staff will use these assessment tools to identify those students who are 'at risk' and then adjust the literacy program to meet their individual learning needs.
The evaluation team is very confident that when these recommendations are implemented, L.B. Primary School’s Guided Reading program will flourish.
Standards
The standards for a successful evaluation, including utility, accuracy, feasibility, propriety, and accountability (Alkin, 2010) reflect the values indicated by our stakeholders in a number of ways. Of primary importance to our stakeholders was that the results of this evaluation have utility; that is, that the results be relevant and help them to implement program improvements. Throughout this participatory evaluation, primary stakeholders and evaluators monitored the accuracy of data. Evaluation conclusions and decisions were jointly arrived at through a dialogic process, thus ensuring sensitivity to the culture and context in which they would be applied. The evaluation team jointly ensured feasibility of the study, as they collaboratively developed a plan that would work within the unique setting of L. B. Primary School. Existing practices and knowledge (such as familiarity with the DRA, and having teachers administer assessments) were built into the design, demonstrating sensitivity and responsiveness to the way in which the program was currently operating. Issues of propriety were addressed through a participatory evaluation design that was responsive to our stakeholders at all stages of the process. The evaluation agreement took into account the needs and expectations of our stakeholders and was arrived at through a process of negotiation. Finally, this evaluation was fully documented, including negotiated purposes, design, procedures, data, and outcomes, thus ensuring full accountability (Alkin, 2010, citing Yarbrough, Shulha, Hopson, & Caruthers).Analysis and Synthesis
The analysis and synthesis of the evaluation findings reflect rigorous examination of the data through the emergence of multiple sources of corroborating evidence. By using various qualitative and quantitative measures, the evaluators were able to collect data and proceed with confidence that it reflected the true state of the L.B. Primary School Guided Reading program. Through the collection, analysis, and interpretation of triangulated data, the evaluators were able to depict and support with evidence the dominant themes that emerged, such as the infidelity of program implementation, time constraints, lack of assessment driven instruction, poor accessibility to resources and training, and a lack of student engagement. Our evaluation conclusions were based on multiple sources of confirmation, providing a detailed and comprehensive picture of the program, and contributing to the trustworthiness of the study.
Interpretation
With our analysis and synthesis of the evaluation findings in mind, the evaluation team reexamined the questions that guided this program.
We examined our over arching question: Is Guided Reading Effective?
The evaluation team also examined the sub questions, which were integral to the program evaluation
and outlined by Fountas & Pinnell (1996)
in reading skills over the course of the school year?
Based on the evidence of the program evaluation and in collaboration with our stakeholders, we concluded that the GR program at L. B. Primary School is not being implemented with fidelity and some students are improving while others are not. The practical significance of what has been learned is that it will be necessary to take steps to improve the guided reading program at L. B. Primary School in order to make it more effective. The stakeholders will be most affected by these means of improvement. After meeting with stakeholders, it was evident that they are wanting practical solutions so that the Guided Reading Program will be effective. Stakeholders want to feel empowered and successful by delivering a guided reading program effectively and faithfully. They want students to improve academically.
Judgments
Fountas and Pinnell state that GR leads to independent reading and gives children an opportunity to grow as individual readers. It provides the readers with an opportunity to develop and apply reading strategies so that they can independently read increasingly difficult texts and to read for meaning. Guided reading develops skills needed for independent reading, and assists in teaching students to learn how to introduce texts to themselves. It is part of a balanced literacy program which measures fluency and reading comprehension by using diagnostic and developmental reading assessments, daily observations, and gathering of anecdotal records. Teachers gather observational data over time to document progress which provides information to guide daily teaching and tracks the progress of individual students. This information is a basis for reporting to parents, helps a school staff to assess the effectiveness of the instructional program, and provides children with evidence of their growth. Guided reading also encourages home and community involvement by assisting parents to participate in the school curriculum, allowing opportunities for children to show their families what they are learning, and increasing reading opportunities for students.
The evaluation indicated that this program did not achieve the intended results due to various contributing factors. Some of the factors that prevented the program from reaching the desired success included staff members lacking the time and skills to deliver the program, limited funds to purchase leveled texts and other materials to allow for the continuation of the program, and assessment data not being used regularly. Overall, teachers felt dissatisfied with the program and therefore were reluctant to continue. However, this does not indicate that the staff was solely responsible for not achieving the intended results. The results from the assessments performed indicated that some students who participated in the GR program did demonstrate improvement in reading skills over the course of the year, while others did not.
Therefore, although the program fell below the established standards, those involved in its delivery felt that it could be extremely effective in meeting the long-term goal of improving student reading skills if it was implemented with fidelity.
Recommendations
The evaluation team met with stakeholders and Board staff to discuss the recommendations prior to publication. The team asked the stakeholders if the recommendations were feasible. The stakeholders were concerned about the students’ limited progress therefore they requested the evaluation even though they were hostile to the process. The stakeholders' goal is for student achievement to improve therefore they were receptive to the recommendations. During the discussion, the stakeholders made a few recommendations based on the evaluation team’s findings. The agreed upon recommendations are presented below.
Based on our findings from the DRA, DIBELS, teacher interviews, and student focus groups, we propose the following recommendations:
The evaluation team is very confident that when these recommendations are implemented, L.B. Primary School’s Guided Reading program will flourish.